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AN  INTRODUCTORY  NOTB^/ 


A.  W.  KArshAlI 


I .  Introduction 

This  Introductorj  note  is  intended  to  fulfill  severel  objectives.  First, 
the  sjaposiua  itself  does  not  contein  »  peper  sussserizin^  pi*ogress  end 
prsctice  in  the  field  of  Monte  Carlo  during  the  period  1949  (the  date  of  the 
last  Monte  Carlo  sTKposiua)  to  the  present.  This  introduction  is  devoted  in 
part  to  filling  this  gap.  Second,  rysposia  usually  present  a  soaewhat  biased 
view  of  the  state  of  any  field  because  papers  are  devoted  to  new  ideas  rather 
than  description  of  standard  practice.  This  would  not  be  an  important  defect 
if  only  experts  in  the  field  were  the  audience  but  there  is  considerable 
evidence  of  a  new,  wider  interest  in  Monte  Carlo.  In  the  case  of  the  present 
syaposiun  the  bias  it  sonewhat  less  than  is  usual.  Many  of  the  papers  give 
applications  which  should  by  now  be  standard  practice  in  Monte  Carlo  coapu- 
tations,  but  are  not  yet  so,  rather  than  new  ideas  or  advanced  techniques. 
However,  there  is  Just  enough  of  the  flavor  of  experts  talking  to  experts  to 
suggest  that  soae  guidance  for  other  than  expert  audiences  would  be  worth¬ 
while.  This  introductory  note  is  primarily  addressed  to  that  growing  class 
of  persons  interested  in  Monte  Carlo  as  a  tool  of  nuaerical  analysis  rather 
than  as  an  area  of  professional  work.  In  other  words  it  is  addressed  to  those 
who  are  not  now  experts  in  the  field  but  have  problesis  which  they  believe 
Monte  Carlo  techniques  say  help  to  solve. 


^  The  author  wishes  to  acknowledge  the  large  debt  he  owes  to  Mr.  Heraan 
Kahn  for  long  discussions  of  loaterial  in  this  note,  and  to  Mr.  R.  T.  Nichols 
for  aany  helpful  suggestions. 
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Vithin  th«  coapass  of  so  short  a  nota  it  is  not  possible  to  conpletely 
reriaw  the  progress  of  Monte  Carlo  or  to  point  up  the  isportant  aspects  of 
each  paper.  The  first  section  following  upon  this  one  is  a  coBoentary  on  the 
progress  of  Monte  Carlo  froa  the  late  forties  to  the  present.  The  second 
section  following  is  devoted  to  soae  explanatory  remarks  on  the  papers 
collected  in  this  voluae. 

Before  going  on,  a  word  about  the  definition  of  Monte  Carlo,  about  which 
there  is  a  good  deal  of  disagreeaent.  The  original  von  Neuaann-Ulax  concept 
seeas  to  have  been  that  Monte  Carlo  specifically  designated  the  use  of  randoc 
saapllng  procedures  for  treating  detenainistic  siathematical  proble&s.  Sosa 
define  Monte  Carlo  to  be  the  use  of  random  sanpling  to  treat  problems,  whether 
of  a  deterainistic  or  probabilistic  sort.  Others  demand  that  the  sampling  be 
sophisticated  (involves  the  use  of  soae  variance  reducing  technique  or 
"swindle")  in  order  to  qualify  as  Monte  Carlo;  they  reserve  the  names  straight 
forward  sampling,  experimental  sampling,  or  oiodel  sampling  for  the  cases  where 
purely  random  sampling  is  used.  The  writer *s  sympathies  are  with  the  latter 
view.  However  the  economics  of  computing  is  changing  so  rapidly  with  the 
advent  of  faster  and  faster  machines  that  sophistication  sometimes  has  a  high 
price.  The  test  of  whether  it  is  Monte  Carlo  rather  than  model  sampling, 
p>erhaps,  might  be  whether  the  sampling  design  was  rationally  chosen;  the 
possible  application  of  the  most  useful  variance  reducing  techniques  always 
being  considered  in  planning  the  treatment  of  any  problem.  As  far  as  the 
definitional  problem  is  concerned,  however,  most  all  discussion  is  beside  the 
point.  Because  of  its  picturesque  character  and  charm  the  use  of  the  name 
Monte  Carlo  has  a  momentum  all  its  own;  the  situation  now  is  that  in  cosaon 
usage  Monte  Carlo  is  synonymous  with  any  uee  of  random  sampling  in  treatment 
of  either  deterministic  or  probabilistic  problems. 
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II.  Th«  Progrtss  of  Mont*  Carlo,  1949-1954 

Th«  following  hiltor/  and  coaMntarj  Mk««  no  pr«t«ns«  of  cocplotonoss. 
Dates  are  only  approxiaate,  as  is  sufficient  for  the  purpose  of  this  section 
—  to  describe  the  general  progress  of  Monte  Carlo  froa  the  late  forties  and 
its  current  state. 

A.  Early  Dominance  of  the  Analogue  Idea 

If  one  looks  at  the  Monte  Carlo  literature  of  the  late  forties  he 
finds  that  it  is  dominated  by  the  intriquing  idea  discovered  or  proauXgated 
by  von  Neunann  and  Ulaa,  that  one  can  use  random  sampling  methods  to  solve 
determinate  mathesmtical  problems.  Others  had  earlier  seen  that  such  things 
could  be  done  in  some  cases,  for  example,  the  calculation  (estimation)  of  t 
by  random  drops  of  a  needle.  But  they  vere  certainly  the  first  to  advocate 
the  idea  of  systematically  inverting  the  ueual  situation  and  treating  mathe¬ 
matical  problems  by  finding  a  probabilistic  analogue  and  then  obtaining 
approximate  answers  to  this  analogue  by  some  experimental  sampling  procedure* 
Their  work  and  discussion  of  the  basic  idea  stimulated  widespread  interest  in 
the  use  of  random  sampling  procedures  in  physics  and  engineering  problems. 

They  were  also  the  first  to  point  out  the  special  adaptability  of  the  high 
speed  computing  machines,  soon  to  be  available,  to  Monte  Carlo  type 
cosputpA^^ons  • 

The  statisticians  had,  of  course,  been  using  model  sampling  methods 
to  investigate  some  of  their  problems,  for  example,  the  effect  of  non¬ 
normality  on  statistical  test  procedures  devised  for  samples  from  normal 
(Gaussian)  populations,  since  the  early  1900*8.  Their  use  of  sampling  reached 
a  peak  in  the  period  1925-1935  and  then  died  off.  However  their  work  was 
concerned  with  probabilistic  problems  so  that  they  were  not  interested  in 
the  sort  of  thing  which  might  lead  to  the  original  von  Neumann-Ulam 
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Idta.  Mor«OT«r,  despite  their  injunctions  to  all  other  experiaental 
professions  that  sajsplin/;  design  was  crucial  in  siaxinieing  the  aaount  of 
inforaation  in  a  saaple  for  a  fixed  expenditure  of  effort,  their  own  work 
shows  no  atteapt  at  design  at  all.  Whatever  ideas  in  sample  survej  design 
that  sdght  have  been  carried  over  at  this  tice  were  not  used  until  the  past 
few  jears.  In  any  case,  the  statisticians  did  not  have  the  analogue  idea 
and  this  is  %fhat  got  Monte  Carlo  in  its  current  Tors  started.  In  addition, 
they  did  not  initiate  the  use  of  variance  reducing  techniques  in  the  area  of 
artificial  sampling  or  model  sampling.  Their  work  in  this  area  represents  a 
sep>arate  stream  that  only  today  is  merging  with  the  much  richer  and  larger 
one  generated  by  the  von  Neumann-Ulam  Monte  Carlo  Ideas.  It  is  always  hard 
to  decide  questions  of  priority  and  direction  of  influence,  but  in  the  case 
of  the  current  development  of  Monte  Carlo  the  situation  is  relatively  clear. 

In  spite  of  the  fact  that  the  von  Ncumann-Ulam  ideas  have  in  a  way 
been  very  fruitful,  it  is  possible  to  get  an  exaggerated  notion  of  the 
practical  importance  of  the  analogue  idea  from  what  has  Just  been  written 
above  and  especially  from  the  literature  of  the  late  forties.  For  some 
problems,  for  example,  particle  diffusion  problems,  the  application  of  this 
idea  meant  a  retranslation  of  problems  back  to  the  probabilistic  basis  they 
had  in  physical  theory  in  the  first  place.  The  success  of  the  idea  was  no 
accident  in  these  cases.  Few,  if  any,  basically  detersdnistic  problems  have 
really  been  treated  by  Monte  Carlo  methods.  Actually  the  dominance  of  the 
literature  by  the  analogue  idea  is  probably  due  more  to  its  novelty  and  its 
great  potentialities  than  to  its  historical  achievements,  and  to  the 
professional  interest  of  most  of  the  people  contributing  papers  —  theoretical 
numerical  analysis  rather  than  applied  problems  of  imrjediate  practical  impor¬ 
tance.  The  use  of  Monte  Carlo  methods  of  computation  have  been  Influenced  by 
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the  analogue  Idea  cainlj  in  an  indirect  waj.  The  aioat  iaportant  practical 
applications  thus  far  have  had  a  probabilistic  basis;  the  influence  of  the 
original  Monte  Carlo  idea  has  been  to  suggest  treating  then  directlj  as 
probabilistic  problems  rather  than  attespting  a  difficult,  if  not  iapoasible, 
analytical  solution.  The  translation  and  later  retranslation  of  probleas 
from  probabilistic  tenes  to  non-probabilistic  aatheoatical  probleau  and  back 
again  has  been  by-passed. 

B.  Early  Applications  of  Monte  Carlo 

Because  the  tsost  iaportant  applications  have  been  on  basically 
probabilistic  problems,  there  is  a  special  interest  in  aentioning  the  iapor¬ 
tant  probleas  vhere  the  pure  analogue  idea  has  already  found  soae  application. 
Around  1947-49  Femi,  with  Metropolis  and  Ulaa,  used  it  for  obtaining  estiaates 
of  the  eigen  values  associated  with  the  Schrodinger  equation.  A  little  later 
Kacs  also  used  Monte  Carlo  Mthods  when  working  on  the  saae  problea.  Work  by 
Metropolis  and  Ulan  is  continuing  at  Los  Alaaos  on  the  application  of  Monte 
Carlo  aethods  to  the  Schrodinger  equation.  However,  thus  far  no  published 
results  show  any  marked  iaprovenents  over  results  attainable  by  clasaical 
techniques. 

About  1930  aatrix  inversion  and  solution  of  partial  differential 
equations  by  Monte  Carlo  methods  were  developed.  Here  again,  although  these 
are  areas  in  which  the  analogue  idea  would  find  use,  no  results  in  cases  with 
ij^nrtant  applications  have  been  published* 

The  firet  practical  proble«s  treated  by  the  NonU  Carlo  aathed  were 
connected  with  the  design  of  atomic  weapons  at  Los  Alamos  during  the  war.  In 
these  problems  nature  was  directly  aodeled  in  its  probabilistic  aspects  and  many 
problems  in  particle  diffusion  were  solved.  Since  these  probleais  were  first 
posed  in  terms  of  solving  complicated  mathematical  equations  to  obtain  expected 
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v&lu«9  the  use  of  nature’s  aodel  of  the  seise  diffusion  process  es  the  enelogue 
problem  Bight  be  considered  es  en  application  of  the  analogue  idea.  It  is  the 
author’s  understanding  that  Ulaa  does  consider  it  so.  However,  once  one 
reaeBbers  that  the  answers  to  the  problea  %t  hand  are  the  expected  values  of 
the  stochastic  process  of  central  inters;  t  in  the  problem,  the  use  of  that 
process  in  obtaining  estimates  of  the  answer  does  not  scea  to  involve  anything 
sufficiently  removed  froa  the  problea,  as  nature  poses  it,  to  be  called  an 
analogue.  Ko  matter  what  it  is  called,  however,  given  that  the  abstract 
foraulation  of  probleas  often  obscures  their  true  basis  the  use  of  nature’s 
Bodel  was  a  great  discovery  and  very  successful.  Later  the  physicists 
extended  the  use  of  these  aethods  of  calculation  to  other  areas,  such  as 
particle  diffusion  in  shields,  aeson  cascade  problers,  etc. 

At  an  early  date  people  working  in  the  general  area  of  operations 
analysis  also  took  up  the  use  of  the  Monte  Carlo  method  in  their  problems. 

This  use  is  now  widely  developed  and  will  be  described  in  aore  detail  below. 

C.  Initial  Developisent  of  Sampling  Technique 

Given  that  one  is  to  do  problems  by  random  sampling  methods, 
attention  naturally  turns  to  three  topics:  (1)  choosing  or  modeling  the 
probability  process  to  be  sampled  (in  some  cases  this  means  choice  of  the 
analogue;  in  others  a  choice  between  alternative  probability  models  of  the 
same  process),  (2)  deciding  how  to  generate  random  variables  from  given 
probability  distributions  in  some  efficient  way,  and  (3)  variance  reducing 
techniques,  l.e.,  ways  of  increasing  the  efficiency  of  the  estimates  obtained 
froa  the  sampling  process.  On  the  first  topic  little  will  be  said.  With 
regard  to  the  second  topic  von  Neumann  was  very  ingenious  from  the  start  but 
general  techniques  had  not  been  worked  out  in  the  early  years.  Indeed  the 
literature  contains  some  Incorrect  statements  of  the  sort  —  it  is  impossible 
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to  do  such  and  such  —  things  which  actuallj  can  b«  tatilj  dona*  Work  Is 
continuing  in  this  araa  and  good  aathods  for  many  distributions  ara  availabla* 
Of  coursa  with  tha  incraasad  spaad  of  tha  conputing  sMchinas  tha  problam  has 
shlftad  a  littla;  tha  asiphasis  is  lass  on  afficiancy  and  aora  on  Just  having 
soaa  wtj  of  ganarating  tha  nacassarjr  randoai  variablas. 

In  tha  case  of  tha  third  topic,  varianca  reducing  techniques, 
von  Nauaann  and  Ulaa  invented  tha  Russian  Roulette  and  splitting  ideas  about 
1945  (sea  Kahn*s  paper)  but  there  was  no  sfstaaatic  attaapt  to  investigate 
this  problan  until  T.  E*  Harris  and  Kahn  began  their  work  in  1948*  Since  than 
a  great  deal  of  progress  has  bean  siada  but  tha  results  ara  not  wall  known,  nor 
widelj  used*  One  hopes  that  tha  publication  of  this  volujca  will  do  soaathing 
toward  correcting  this  deficiency  in  tha  currant  practice  of  Monte  Carlo. 

0*  Tha  Current  State  of  Monte  Carlo 

Since  tha  lata  40*s  tha  siain  trends  have  bean: 

1.  A  relative  decline  of  interest  anong  people  purely  concerned 
with  technique  and  especially  in  the  interest  in  the  exploration  of  the 
pure  analogue  idea.  Tha  developsvnt  of  some  first  rate  applications  of 
tha  idea  is  probably  required  to  stiDulate  further  work  in  this  now 
more  or  less  donaant  area. 

2*  A  very  great  absolute  and  relative  increase  in  applied  use  of 
Monte  Carlo,  aostly,  as  has  been  reaarked,  to  problens  with  a  proba¬ 
bilistic  basis.  Overt  interest  in  Monte  Carlo  probably  passed  through 
a  sdninue  in  1951-52;  the  novelty  had  worn  off  in  the  area  of  theoretical 
interest  and  applied  use  was  not  as  widespread  as  it  now  is. 

3*  The  statisticians,  who  all  along  have  been  by  professional 
training  esdnently  qualified  to  siake  important  contributions,  have  now 
becoae  interested  in  tha  field,  both  for  use  on  their  own  problesis  and 
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fcs  a  ftparalt  profasiional  area  of  taapling  tachnlqut.  Harris  and  a 
faw  olhart  w«r«  intsrestad  in  and  workinf  in  the  field  aLaott  froa  the 
verj  beginning,  but  it  is  onlj  lateljr  that  articles  on  Monte  Carlo  have 
begun  to  appear  in  statistics  Journals.^ 


^  The  devclopaent  of  Monte  Carlo  independently  of  the  statisticians  and 
particularly  their  poor  showing  in  applying  saapling  techniques  to  their  own 
probleais  say  be  surprising  to  soae;  especially  to  those  who  think  Student 
invented  Monte  Carlo.  Soae  writers  in  the  field  of  Monte  Carlo  have  been 
excessively  polite  and/or  generous  in  indicatixvg  that  many  or  all  of  the 
variance  reducing  ideas  coae  froa  survey  sasipling  theory.  This  is  not  true 
since  Most  of  the  iaportant  ideas  exploit  the  extra  degrees  of  freedom  avail¬ 
able  in  the  design  of  sampling  procedures  that  result  from  the  fact  that  one 
has  coaiplete  control  of  the  generation  and  selection  of  the  random  variables 
he  uses.  What  is  surprising  is  that  the  statisticians  did  not  take  the  advice 
they  gave  to  others  on  the  use  of  efficient  designs. 

As  to  the  invention  of  Monte  Carlo  by  Student,  it  is  a  strange  case. 
Student  did  something  slightly  different,  but  much  better  than  most  Monte  Carlo 
calculations.  In  the  case  of  his  discovery  of  the  t  distribution  he  had  de¬ 
rived  the  correct  distribution  analytically,  but  he  had  i^de  some  jumps  in  the 
logic.  Being  unsure,  but  not  very  unsure,  of  the  derived  distribution  he 
tested  it  upon  some  samples  he  had  drawn  for  exploratory  purposes,  computing  t, 
and  testing  the  sample  distribution  against  the  theoretical  using  a  test. 

He  states  that  he  first  tried  to  get  the  distribution  by  sampling,  but  how  he 
used  the  saaq>ling  results  to  suggest  the  right  answer  is  not  indicated.  In  his 
paper  the  sample  results  are  used  only  as  a  check. 

In  the  case  of  the  distribution  of  the  correlation  coefficient  Student 
knew  that  for  n*2  the  distribution  degenerate  and,  in  addition,  he  had  some 
samples  for  p  *  0  of  site  4  and  8  and  for  p  ■  .66  of  site  4,  6,  and  30.  from 
the  sam^''e  distributions  and  moments  for  the  case  p  •  0  he  conjectured  that  a 

r  2  -iO(n) 

Pearson  distribution  of  the  for*  f(x,n)  •  k  j^l  -  •sr^J  would  approximate  the 

distribution.  Fitting  a  and  p  for  n«4,  6  he  rounded  and  guessed  that  a  ” 

and  P  •  1|  checking  that  for  n*2  the  distribution  fitted  exactly.  Thus  in  ^ 

the  cases  of  both  t  and  r.  Student  made  magnificent  guesses,  partly  suggested 
by  prior  samples,  and  supported  by  checking  against  actual  samples  and  analytic 
results  for  large  sample  approximations  of  the  moments  of  the  distributions, 
etc.  All  of  this  isay  or  may  not  be  Monte  Carlo,  it  is  different  from  8X>3t 

applications,  and  in  any  case  an  isolated  instance  of  first  rate  use  of  sampling 

for  statistical  purposes. 

To  the  extent  the  later  sampling  experiments  of  the  statisticians  were 
multi-purposed  this  would,  of  course,  inhibit  the  application  of  variance 
reducing  techniques,  since  this  requires  one  to  select  what  it  is  that  is  of 
most  interest  and  to  concentrate  upon  the  reduction  of  the  variance  of  that 
particular  characteristic,  often  at  the  expense  of  increasing  the  variance  of 
the  others.  Interest  in  generating  data  giving  unbiased  information  about 
many  aspects  of  a  problem  also  makes  it  difficult  to  apply  variance  reducing 
techniques  to  many  operations  research  problems  as  they  tend  to  be  stated.  But 
in  investigating  such  things  as  the  effect  of  non-normality  or  comparisons  of 
the  power  functions  of  tests  powerful  techniques  have  long  been  available. 
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A  few  new  areas  for  the  application  of  Monte  Carlo  have  been  opened 
up  In  the  past  few  years.  The  aost  iaportent  of  these  is  statistical  aechanlcs. 
Here  Metropolis,  Rosenbluth  and  Teller  pointed  out  that  one  can  exploit  eaapling 
processes  possessing  the  right  kind  of  ergodic  properties.  These  saopling 
processes  allow  one  to  reproduce  (converge  to)  the  correct  equilibrlu*  distri¬ 
bution  of  various  quantities  such  as  the  total  energy  of  a  cubic  lattice 
system.  At  least  two  groups  are  now  working  to  exploit  these  ideas. 

There  has  also  been  an  increased  use  of  Monte  Carlo  methods  in 
operations  analysis  problems  especially  as  these  have  become  store  comprehensive 
and  richer  in  detail  and,  therefore,  beyond  the  scope  of  analytic  and  classical 
numerical  sMthods.  In  this  case  Monte  Carlo  is  used  almost  by  default,  since 
the  only  feasibility  lifcdt  to  Monte  Carlo  is  one  of  energy  or  computing  power. 

As  has  been  mentioned,  the  advent  of  more  and  faster  coaputing  machines  has 
had  its  impact  on  Monte  Carlo  just  an  it  has  on  other  numerical  analysis 
techniques.  This  technological  growth  has  led  to  a  tendency  to  do  larger  and 
larger  problems,  which  if  they  have  a  probabilistic  basis  has  tended  to  make 
Monte  Carlo  a  preferred  method  of  analysis. 

The  increase  in  the  speed  of  the  machines  has  also  tended  to  make 
variance  reducing  techniques  relatively  less  interesting,  but  has  by  no  means 
eliminated  their  usefulness.  The  effect  of  Increased  computing  speed  in  the 
newer  machines  is  to  make  the  cost  of  designing  and  coding  a  problem  increase 
relative  to  the  cost  of  machine  running  time.  The  use  of  variance  reducing 
techniques  shortens  running  time  but  at  the  expense  of  (1)  Increasing  the  time 
spent  in  designing  the  computations  so  as  to  adapt  the  classical  techniques 
to  the  particular  problem  or  in  the  invention  of  a  new,  more  suitable  technique, 
and  (2)  complicating  the  coding  because  of  the  more  elaborate  bookkeeping  and 
calculations  these  techniques  usually  require.  On  the  whole,  however,  if 
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there  is  one  thing  that  would  generally  increase  the  usefulness  of  Konte  Carlo 
it  la  the  discovery  of  new  variance  reducing  techniques,  or  the  application  of 
known  variance  reducing  techniques  as  a  catter  of  course  to  the  ordinary  nin 
of  problems.  Not  only  should  these  techniques  be  used  whenever  it  is  econoedcal 
to  do  so  but,  in  addition,  since  the  variance  reducing  techniques  are  not  yet 
well  known  there  should  be  a  bias  toward  using  thea  even  when  they  are  not 
economical  for  the  problem  at  hand  in  order  to  learn  about  thea  for  use  in 
later  problems.  The  use  of  new  techniques  in  marginal  cases  is  almost  always 
Justified  as  a  method  of  building  intellectual  capital.  In  the  long  run  one 
would  suppose  that  real  thought  on  the  design  of  Monte  Carlo  problems  will  be 
confined  to  problems  of  a  basically  new  type  whenever  they  first  appear; 
standard  variance  reducing  techniques  will  be  available,  and  used,  for  other 
problems  on  thf  basis  that  sub-routines  for  computing  common  functions  now 
are. 

III.  Comments  on  the  Papers  in  the  Volume 

In  this  section  cosoments  upon  the  papers  contained  in  this  volume  will 
be  given.  The  aim  of  the  comments  is  oalnly  to  assist  those  not  now  expert 
in  the  field  of  Monte  Carlo  to  get  a  little  more  than  they  sdght  otherwise 
out  of  reading  this  collection  of  papers. 

The  papers  are  grouped  for  discussion  into  three  groups:  (1)  those 
dealing  with  the  generation  of  random  numbers  and  more  general  random 
variables;  (2)  theoretical  papers;  and  (3)  applied  papers.  Not  all  papers 
will  be  commented  upon,  in  particular  papers  included  only  as  abstracts 
will  not  be  conaented  upon. 

A.  The  Generation  of  Random  Numbers  and  More  General  Random  Variables 
Oie  of  the  first  problestis  faced  by  anyone  doing  Monte  Carlo 
calculations  is  the  generation  of  the  sequences  of  random  variables  required 


P-756 

10-24-55 

-11- 


by  the  problea.  This  generally  breaks  down,  If  the  problec  Is  to  be  run  on  a 
high  speed  coaputing  Machine,  into  two  problems :  (1)  how  to  generate  sequences 

of  random  numbers  uniformly  distributed  on  the  interral  (0,  1);  and  (2)  how  to 
transform  the  randon  numbers  into  random  variables  haring  specified  probabilitj 
distributions.  Previously  prepared  tables  of  random  numbers  and  special 
random  variables,  e.g.,  random  normal  deviates,  are  also  useful,  espccialljr  if 
they  are  on  p\inch  cards  or  tapes  suitable  for  feeding  into  computing  machines. 
However,  first  rate  methods  are  now  available  for  the  generation  of  random 
variables  in  the  very  large  numbers  needed  for  the  types  of  problems  run  on 
high  speed  computing  machines;  tables  are  of  real  help  only  with  hand  compu¬ 
tations  (or  computations  with  traditional  IHM  type  equipment). 

The  papers  of  Butler,  Lytle,  Metropolis,  and  Taussky-Todd  deal  with 
various  aspects  of  the  problem  of  supplying  random  variables  for  use  in  Monte 
Carlo  computations.  The  Taussky-Todd  paper  is  an  especially  good  summary  of 
the  currently  available  and  tested  methods  of  generating  sequences  of  pseudo¬ 
random  numbers  in  high  speed  computing  suchines.  The  bogus  character  of  the 
randomness  of  the  generated  sequences  does  not  seem  to  affect  the  Monte  Carlo 
calculations.  Just  as  it  is  not  detected  by  the  various  statistical  tests  to 
which  these  sequences  have  been  subjected,  and  through  which  they  have  passed 
with  flying  colors.  The  congruential  methods  •  K  X^(Mod  M))  of  gener¬ 

ating  pseudo-random  nuebers  are  so  good  for  practical  computational  purposes 
that  there  no  longer  exists  any  problem  of  having  an  adequate  supply  of  random 
numbers.  Thus  the  discovery  of  newer  methods,  such  as  the  use  of  the  Fibonacci 
sequence,  is  no  longer  practically  important,  although  it  may  be  of  great 
theoretical  interest. 

The  Metropolis  paper  contains  a  very  interesting  section  on  the 
mid-square  method  of  generating  random  numbers.  This  was  the  first  random 
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nuxber  generating  Method  adepted  for  uae  on  high  speed  Machines.  It  is  not, 
however,  as  good  a  nethod  of  generating  randojs  nuiabers  as  the  newer  congx*u- 
ential  Methods:  the  two  aethoda  take  about  the  saae  tise  to  produce  a 
randoa  number  (cost  the  same),  but  for  the  newer  Machines  sequences  of  period 
on  the  order  of  are  possible  with  the  congruent  Methods  no  matter  what 

starting  value  Is  used.  On  the  other  hand.  Metropolis’  best  starting  value 
gives  a  sequence  of  length  (or  period?)  on  the  order  of  10^  and  for  taosl 
other  starting  values  the  sequences  are  probably  much  shorter,  on  the  order 

A 

of  10  .  It  is  a  substantial  practical  advantage  of  the  congruent  method  that 
one  does  not  have  to  pick  a  special  initial  value  in  order  to  get  long 
sequences  of  usable  numbers.  Konetheless  the  study  of  the  characteristics 
of  sequences  of  random  numbers  generated  by  the  mid-square  method  is  of 
classical  interest. 

The  Lytle  paper  provides  an  example  of  the  second  problem  mentioned 
above  —  the  transformation  of  random  numbers  into  random  variables  with  the 
appropriate  probability  distributions.  The  Butler  paper  deals  more  generally 
with  this  problem  and  describes  in  some  detail  a  few  of  the  rsore  Interesting 
methods  of  achieving  the  required  transformations.  This  is  an  area  in  which 
ingenuity  is  often  required  for  success.  It  is  only  gradually  that  there 
has  developed  a  practical  capacity  to  generate  randoa  variables  from  a  wide 
class  of  probability  distributions.  The  choice  of  any  particular  scheme  of 
course  depends  upon  the  economics  of  the  computation  situation.  For  example, 
at  RAND  we  currently  generate  randoa  variables  having  the  exponential  distri¬ 
bution  by  taking  the  logarithm  of  random  numbers,  i.e.,  by  the  direct  method. 
Excellent  approximations  of  In  X  are  available  for  use  on  modern  high  speed 
machines  that  make  this  scheme  competitive  with,  and  probably  more  efficient 
than,  rejection  techniques. 
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3.  Theor«tic4l  Papers  on  Monte  Carlo 

Tha  papers  concamed  aalnly  with  theoretical  aspects  of  Honte  Carlo 
methods  arc  those  of  Albert,  Curtiss,  Kahn,  Karshall  (with  Walsh’s  cosenant), 
Motzkin,  Trotter  an^4  Tukej,  and  Walsh.  No  comsnt  will  be  made  about  the 
Motikln  or  Walsh  papers.  The  remaining  papers,  with  the  exception  of  Curtiss* 
paper,  deal  with  the  central  probleei  of  rarlance  reduction. 

Albert’s  paper  Is  concerned  with  the  application  of  Monte  Carlo 
methods  to  certain  integral  equation  problems.  Several  alternative  methods 
of  producing  estiJiates  are  suggested  and  discussed  from  the  point  of  view  of 
the  relative  size  of  their  variances.  The  extent  to  which  these  particular 
estimates  nave  been  used  is  not  mentioned  in  the  paper. 

Kahn’s  paper  reviews  the  techniques  of  variance  reduction  that 
have  thus  far  proved  to  be  most  useful  in  Monte  Carlo  computations.  Kany  of 
these  techniques  were  first  developed  by  Kahn,  or  at  least  perfected  and 
used  in  important  applied  Monte  Carlo  problems  for  the  first  time  by  him. 

My  own  paper  Investigates  a  more  or  loss  mechanical,  two-stage 
sampling  method  for  choosing  good  imp,  'tance  sampling  schemes.  The  technique 
discussed  is  fairly  straight foirward  but  must  be  considered  as  wholly  untried. 
Some  technical  problems  arise  in  the  paper  because  the  expected  value  of  the 
variance  of  the  estimate  derived  from  the  second  stage  in  the  sampling  scheme 
is  infinite,  at  least  in  a  wide  class  of  cases.  Depending  on  how  large  the 
first  sample  is,  however,  the  variance  of  the  second  stage  estimate  is  smaller 
than  that  of  the  first  stage  with  high  probability.  Walsh  makes  the  very 
reasonable  comnent  that  in  this  case  one  might  work  with  other  measures  of 
variation  than  the  variance  in  order  to  avoid  analytic  problems.  This  is 
not  the  only  way  out,  but  is  a  good  one  if  the  other  measures  prove  more 
tractable  analytically  and  in  applied  numerical  work. 
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The  Trotter  And  Tukej  paper,  with  its  coEpenlon  applied  paper 
nentioned  in  the  next  section,  is  probeblj  the  r.08t  exciting  paper  in  the 
Toluae.  First,  the  authors  Introduce  a  new  rariance  reducing  technique 
(conditional  Monte  Carlo)  and,  second,  in  the  applied  paper  they  show  that 
it  works  very  well.  It  is  heartening  to  see  nathenatical  statisticians 
really  beconing  Interested  in  Monte  Carlo  techniques  and  using  thea  on  their 
own  problens.  This  paper  deserves  close  study  (and  needs  it)  to  get  the 
conditional  Monte  Carlo  trick  straight.  It  seess  to  be  more  coaplicated 
than,  for  example,  the  aethods  discussed  by  Kahn.  The  paper  also  contains, 
besides  the  technical  gen  already  nentioned,  many  wise  wards  on  the  good 
practice  of  Monte  Carlo  which  reflect  an  attitude  that  should  be  sore 
widespread. 

Curtiss*  paper  is  prljrarily  an  exajelnation  of  the  relative  accuracy 
of  two  classical,  non-stochastic  aethods  and  the  Monte  Carlo  aethod  of  com¬ 
puting  one  component  of  the  solution  of  a  linear  equation  system.  The  paper 
gives  a  first  rate  and  fair  comparison  of  the  methods.  One  ends  with  the 
feeling  that  unless  something  can  be  done  to  Increase  the  accuracy  of  the 
Monte  Carlo  method  in  this  area  that  it  will  remain  merely  an  interesting 
novelty,  especially  when  as  Curtiss  suggests  it  is  so  seldom  the  case  in 
actual  applied  problems  that  only  one  component  of  the  solution  is  required. 

C.  Applied  Monte  Carlo  Papers 

The  applied  papers  are  those  of  Arnold,  Bucher,  Trotter  and  Tukey; 
Beach  and  Theus;  Berger;  and  Disnuke,  The  paper  by  Arnold,  et  al,  is  one  of 
the  first  papers  to  be  published  in  which  substantial  and  impressive  reduc¬ 
tion  in  variance  is  achieved.  Other  problems  have  been  done  as  well,  and 

6 

indeed  factors  of  10  in  variance  reduction  have  been  achieved  in  some  cases, 
but  unfortunately  none  of  these  results  have  been  published  in  a  form  that  is 
both  convincing  and  instructive. 
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The  p*pcr  of  B«Ach  and  Theus  also  helps  to  fill  the  saae  gap  in  the 
literature.  It,  along  with  the  Berger  paper,  is  the  best  thing  yet  published 
in  the  field  of  the  application  of  Monte  Carlo  to  nucleer  shielding  problems. 
This  is  an  area  teherc  some  of  the  best  Monte  Carlo  has  been  done.  Kahn*s 
Nucleonics  paper  describes  the  variance  reducing  techniques  but  numerical 
examples  of  their  use  were  not  presented.  In  some  sense  however  the  work 
behind  the  paper  of  Beach  and  Theus  is  better  than  the  paper  itself.  It 
is  not  as  educational  as  it  could  be.  For  example,  much  of  the  success  in 
choosing  good  variance  reducing  techniques  in  applied  Monte  Carlo  calcula¬ 
tions  has  been  due  to  a  physical  understanding  of  the  problem  or  Intuition 
as  to  the  processes  in  the  problem  which  largely  determine  the  answer.  Thus 
in  the  present  paper  it  would  be  instructive  to  hear  more  about  (1)  the 
rationale  behind  the  choice  of  the  various  sampling  schemes  that  were  tried, 
(2)  why  those  that  worked  well  did  so,  (3)  why  some  combination  of  energy 
and  scattering  biasing  and  the  exponential  transformation  would  probably  be 
required  for  the  treatment  of  problems  with  gama  rays  of  lower  energy,  say 
2  Mev,  etc. 

Berger  uses  somewhat  different  variance  reducing  techniques 
than  those  of  Beach  and  Theus,  i.e.,  correlated  sampling  and  more  sophis¬ 
ticated  methods  of  extracting  Information  from  the  random  walks.  These 
techniques  are  beat  classified  under  the  heading  of  the  method  of  expected 
values  discussed  in  Kahn*s  paper.  Unfortunately  while  the  paper  illus¬ 
trates  the  use  of  the  method  of  expected  values  it  does  not  show  any  real 
advantage  over  straightforward  sampling  in  some  cases,  for  example,  in  the 
estijTvation  of  the  transmission  coefficients.  This  method,  which  often 
does  very  well,  probably  gives  best  results  in  this  type  of  problem  in  the 
range  of  4-6  mean  free  paths.  Below  this  (1-4  mean  free  paths)  it  does  not 
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do  too  well  At  the  pAper  shows.  Bejond  6  neAn  free  pAths  the  sethod  continues 
to  ipcreAsc  In  efficiency  relAtire  to  streightforwArd  sAicpling;  but  not  fAst 
enough  to  hAndle  the  problea  which  is  growing  in  difficulty  by  leeps  And 
bounds.  Doth  methods  Are  swsusped  in  this  CAse.  On  the  other  hAnd,  while 
the  estimAtes  of  At  leAst  some  of  the  coefficients  Are  not  improved  the 
cstimAtes  of  the  angular  spectrum  of  the  emerging  gAssrA  reys  ere  prolAbly 
very  much  improved.  Thus  Any  Judgment  of  the  vAlue  of  the  method  of  expected 
vaIucs  in  this  problem  depends  upon  what  is  of  most  interest,  transmission 
coefficients  or  the  angular  spectrum.  Data  on  the  use  of  correlated  sampling 
is  not  given  although  Berger  has  been  using  It  but  it  is  undoubtedly  very 
effective  in  increasing  the  efficiency  with  which  the  effects  of  changing 
geometries  can  be  studied. 

The  OisBuke  paper  is  cosipletely  straightforward.  It  describes  the 
setup  of  an  applied  problem  of  the  level  of  complexity  where  Monte  Carlo 
really  becomes  Interesting  as  a  numerical  technique. 

IV.  Conclusion 

In  conclusion,  it  siay  be  noted  that  there  Is  considerable  evidence  of 
the  existence  of  a  large  non-expert  Interest  In  Monte  Carlo.  Unfortunately, 
the  literature  on  Monte  Carlo  is  not  very  helpful  to  anyone  who  wishes  to 
do  Konte  Carlo  problems  of  his  own  and  who  looks  for  guidance.  If  only 
through  example,  as  to  how  he  should  best  proceed.  There  are  now  a  number 
of  papers,  including  some  in  the  present  volume,  which  discuss  the  general 
principles  of  good  Monte  Carlo  practice  —  the  most  usually  applicable 
variance  reducing  techniques,  what  to  look  for  in  general,  etc.  However, 
the  Monte  Carlo  literature  Is  almost  completely  devoid  of  well  done  applied 
probleeis,  which  are  essential  to  the  widespread  use  the  techniques.  Well 
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done  probleKs  would  prob«blj  b«  useful  even  if  they  Mrely  raised  standards 
of  aspiration  without  teachinf  much.  Technique  can  be  learned  by  the  study 
of  well  done  problems,  whether  trivial  or  not.  One  of  the  good  things  about 
the  present  volume  is  that  it  does  contribute  a  few  more  examples  of  applied 
Monte  Carlo. 

In  icost  cases  a  little  thought  pays  big  dividends  in  variance  reduction 
and  the  use  of  straightforward  sampling  is  presumptive  evidence  of  a  lack  of 
thought  or  ingenuity  in  design  of  the  sampling.  Some  thought  is  required, 
however,  since  only  a  few  people  have  built  up  the  necessary  intellectual 
capital  required  to  do  problems  well  without  much  effort.  It  is  suggested 
that  readers  who  wish  to  really  understand  the  various  techniques  (importance 
sampling,  systematic  sampling,  stratified  sampling,  and  Russian  Roulette  and 
splitting)  work  through  simple  problems  using  them.  The  basic  ideas  are 
almost  trivial,  but  long  and  disappointing  experience  has  shown  that  they 
seem  very  hard  to  communicate  in  a  useful  way.  Contact  with  problems  seems 
essential  to  understanding  for  most  people.  Because  one  seems  to  be  getting 
something  for  nothing,  it  is  necessary  to  keep  straight  the  process  by  which 
everything  comes  out  all  right  in  the  end;  the  efficiency  of  the  methods  in 
particular  cases  seems  unbelievable.  The  results  quite  literally  have  to 
be  seen,  and  seen  through,  to  be  believed. 
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